Papers with Iterative Nullspace Projection
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. |
| Approach: | They propose to use Counterfactual Data Augmentation, Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebia as bias mitigation techniques to quantify their effectiveness. |
| Outcome: | The proposed techniques are Counterfactual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebia. |
Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE (2025.findings-acl)
Copied to clipboard
| Challenge: | Amnesic probing examines the influence of specific linguistic information on the behaviour of a model by identifying and removing a property and then testing whether the model can still perform well on next-word prediction. |
| Approach: | They propose to use iterative nullspace projection to remove information by iterating on the target property and then assessing whether the model's performance changes. |
| Outcome: | The proposed methods remove information in a more targeted manner, thereby enhancing the potential for obtaining behavioural explanations through Amnesic Probing. |